Digital watermark detection method and apparatus

ABSTRACT

A specific frequency component extraction unit extracts a specific frequency component signal from an input image signal. A phase controller controls the phase of the specific frequency component signal. A correlator computes the cross-correlation value of the phase-controlled specific frequency component signal and the input image signal. A watermark information estimation unit filters the cross-correlation value to detect watermark information embedded in the input image signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present divisional application claims the benefit of priority under35 U.S.C. §120 to application Ser. No. 10/626,610, filed Jul. 25, 2003,and under 35 U.S.C. § 119 from Japanese Patent Application No. No.2002-218404, filed Jul. 26, 2002, the entire contents of both areincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a digital watermark detection methodand apparatus useful in preventing illegal copies of a digital videosignal provided via, for example, a recording medium.

2. Description of the Related Art

As apparatuses for recording and playing back digital image data, suchas a digital VTR, DVD (Digital Versatile Disk), and the like haveprevailed, the number of digital moving images that can be played backby these apparatuses are provided. Various digital moving images aredistributed via digital television broadcast via the Internet, broadcastsatellite, communication satellite, and the like, enabling users toenjoy high-quality digital moving images.

It is easy to form high-quality copies from digital moving images on thedigital signal level. Therefore, if some copy protection or copy controlis not applied to digital moving images, there is the danger ofunrestricted formation of copies of digital images. Therefore, illicitcopies of digital images must be prevented, and the number ofgenerations of copies formed by authorized users must be restricted. Forthis purpose, a method of appending information for copy control to eachdigital moving image, and preventing illicit copies or restrictingcopies has been proposed.

As a technique for superposing additional information to a digitalmoving image in such a way, digital watermarking is known. In digitalwatermarking, information such as identification information of thecopyright owner or user of contents, right information of the copyrightowner, use conditions of contents, secret information required uponusing contents, the aforementioned copy control information, or the like(such information will be referred to as watermark informationhereinafter) is embedded in contents of audio data, music data, movingimage data, still image data, or the like, which has been converted intodigital data, so as not to be easy to perceive. By detecting theembedded watermark information from the contents later as needed,copyright protection, including use control and copy control, can beachieved, and further use of the contents is possible.

As a conventional method of digital watermarking, a method that appliesa spread spectrum technique is known. In this method, watermarkinformation is embedded in a digital moving image in the followingsequence.

In step E1, an image signal undergoes spread spectrum by beingmultiplied by a PN (Pseudorandom Noise) sequence.

In step E2, the image signal after spread spectrum undergoes frequencytransformation (e.g., DCT transformation).

In step E3, watermark information is embedded in the image signal bychanging the values of specific frequency components.

In step E4, the image signal undergoes inverse frequency transformation(e.g., IDCT transformation).

In step E5, the image signal undergoes inversely spread spectrum (theimage signal is multiplied by the same PN sequence as in step E1).

Watermark information is detected in the following sequence, from thedigital moving image, in which the watermark information has beenembedded in the above sequence.

In step D1, the image signal undergoes spread spectrum by beingmultiplied by a PN (Pseudorandom Noise) sequence (the same PN sequenceas in step E1).

In step D2, the image signal after spread spectrum undergoes frequencytransformation (e.g., DCT transformation).

In step D3, the embedded watermark information is extracted from theimage signal while paying attention to the values of specific frequencycomponents.

When digital watermarking is applied to digital productions for thepurpose of prevention of illicit use, a characteristic (robustness) thatcan prevent watermark information from being lost or tampered with, anddeliberate attacks which are normally carried out on digital productionsmust be provided to digital watermarking. As attacks that make thewatermark information of a digital image impossible to detect, cut-out,scaling (enlargement/reduction), rotation, and the like of an image areknown.

When an image that has suffered such attacks is input, the conventionaltechnique recovers synchronization of a PN sequence by executing aprocess for estimating a PN sequence used in step E1 at the time ofembedding upon detection of watermark information. After that, theprocesses in steps D1 to D3 are executed to extract the embeddedwatermark information. However, in order to recover synchronization ofthe PN sequence from the image signal alone, a search must be conductedby trying a process for detecting watermark information using aplurality of candidates of PN sequences and adopting a candidate thatcan be detected satisfactory. For this purpose, problems of increases inarithmetic operation volume and circuit scale are posed. Further, sincewatermark embedded in an image signal under an attack of scaling orrotation is weakened, it is very possible that the watermark cannot bedetected even if the contents (scaling, rotation, etc.) of the attack isdetected and a detection method corresponding to the attack is utilized.

BRIEF SUMMARY OF THE INVENTION

It is an object of the present invention to provide a digital watermarkdetection method and apparatus, which can more accurately detectwatermark information weakened by an attack such as scaling, rotation,etc., without increasing the operation amount and circuit scale.

According to an aspect of the invention, to detect watermark informationembedded in an input image signal, firstly, a specific frequencycomponent signal is extracted from the input image signal. The phase ofthe specific frequency component signal is controlled, and across-correlation value between the phase-controlled specific frequencycomponent signal and the input image signal is computed. The watermarkinformation is detected from the cross-correlation value. A correlationoperation is performed to compute the cross-correlation value, whilechanging the phase control amount, with the result that watermarkinformation can be detected even if the input image signal is under anattack of scaling.

According to another aspect of the invention, to detect watermarkinformation embedded in an input image signal, firstly, theauto-correlation function of the input image signal is computed. Theauto-correlation function is filtered to generate a specific frequencycomponent signal. From this specific frequency component signal, thewatermark information is detected. Before the auto-correlation functionis computed, image rotation may be performed on the input image signal.

According to a further aspect of the invention, to detect watermarkinformation embedded in an input image signal, firstly, theauto-correlation function of the input image signal is computed. Theauto-correlation function is accumulated for a first period of time,thereby generating a first accumulation signal. A specific frequencycomponent signal is extracted from the first accumulation signal, andthe amplitude of the specific frequency component signal is normalized.The normalized specific frequency component signal for a second periodof time longer than the first period of time, thereby generating asecond accumulation signal. From the second accumulation signal, thewatermark information is detected. As in the previous case, imagerotation may be performed on the input image signal before theauto-correlation function is computed.

According to a further aspect of the invention, a computer readablestorage medium storing instructions of a computer program which whenexecuted by a computer results in performance of steps including:computing an auto-correlation function of the input image signal by acorrelator; accumulating the auto-correlation function for a firstperiod of time to generate a first accumulation signal by a firstaccumulator; extracting a specific frequency component signal from thefirst accumulation signal by an extraction unit; normalizing anamplitude of the specific frequency component signal by a normalizingunit; accumulating the normalized specific frequency component signalfor a second period of time longer than the first period of time togenerate a second accumulation signal by a second accumulator; anddetecting the watermark information from the second accumulation signal,wherein at least one of the correlator, the first accumulator, thenormalizing unit, and the second accumulator includes a processor, andwhich further comprising controlling which periodically stopscomputation of the correlator to reduce a operation amount of thecorrelator per unit time, if a throughput of the processor is lower thana threshold value. In addition, the detecting detects the watermarkinformation by determining a polarity of a peak of the specificfrequency component signal. In addition, the detecting detects thewatermark information by determining a level of the second accumulationsignal using a threshold value that is changed in accordance with thesecond period of time. In addition, the detecting detects the watermarkinformation using at least first and second detection manners, thedetecting determining that the watermark information is embedded, if thedetection results coincide with each other.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

FIG. 1 is a block diagram showing a digital watermark detectionapparatus according to a first embodiment of the present invention;

FIG. 2 is a chart for explaining phase shift of a specific frequencysignal by a phase controller in the first embodiment;

FIG. 3 is a graph showing an operation example of peak search for across-correlation value and watermark information detection in thedigital watermark detection apparatus according to the first embodimentof the present invention;

FIG. 4 is a block diagram showing a digital watermark detectionapparatus according to a second embodiment of the present invention;

FIG. 5 is a block diagram showing a digital watermark detectionapparatus according to a third embodiment of the present invention;

FIG. 6 is a block diagram showing an essential part of a digitalwatermark detection apparatus according to a fourth embodiment of thepresent invention;

FIG. 7 is a block diagram showing an essential part of a digitalwatermark detection apparatus according to a fifth embodiment of thepresent invention;

FIG. 8 is a block diagram showing an essential part of a digitalwatermark detection apparatus according to a sixth embodiment of thepresent invention;

FIG. 9 is a view useful in explaining computation of correlationperformed in an oblique direction to detect a digital watermark from animage subjected to rotational transform;

FIG. 10 is a block diagram illustrating a first concrete example of awatermark estimation unit incorporated in the digital watermarkdetection apparatus;

FIG. 11 is a block diagram illustrating a second concrete example of thewatermark estimation unit incorporated in the digital watermarkdetection apparatus;

FIG. 12 is a block diagram illustrating a third concrete example of thewatermark estimation unit incorporated in the digital watermarkdetection apparatus;

FIG. 13 is a view illustrating a determination threshold value forwatermark information detection, which are changed in accordance with anaccumulation period of time;

FIG. 14 is a view illustrating a general correlation operation;

FIG. 15 is a view illustrating a correlation operation performed onevery other pixel;

FIG. 16 is a view illustrating a correlation operation performed oneight pixels contained in every other block; and

FIG. 17 is a view illustrating a correlation operation performed onevery other pixels contained in every other block.

DETAILED DESCRIPTION OF THE INVENTION

Embodiments of the invention will be described in detail with referenceto the accompanying drawings.

First Embodiment

A digital watermark detection apparatus according to a first embodimentof the invention receives, via a recording medium or transmissionmedium, an image signal having watermark information embedded therein,which is generated by a digital watermark embedding apparatus (notshown) corresponding to the digital watermark detection apparatus.

This digital watermark embedding apparatus will now be describedbriefly. In the digital watermark embedding apparatus, a specificfrequency signal extraction unit extracts, from an original imagesignal, a specific frequency component, for example, a relatively highfrequency component. The specific frequency component signal issubjected to phase control, performed by a phase controller inaccordance with a specific phase control amount that is predetermined bydigital watermark information to be embedded into an input image signal.The phase-controlled specific frequency component signal is supplied toa watermark information superposition unit formed of a digital adder,where it is superposed upon the original image signal. As a result, animage signal embedded with watermark information is generated.

The thus-obtained image signal embedded with watermark information isrecorded on a recording medium by, for example, a digital imagerecording/reproducing apparatus, such as a DVD, or transmitted via atransmission medium, such as the Internet, a broadcasting satellite, acommunication satellite, etc.

Such a digital watermark embedding apparatus as described above isdisclosed in, for example, U.S. patent application Ser. No. 10/327,072,the entire contents of which are incorporated herein by reference.

Referring now to FIG. 1, an input image signal 10 embedded withwatermark information is supplied to the input of a specific frequencycomponent extraction unit 11 and the first input of a correlator 13. Thespecific frequency component extraction unit 11 comprises a digitalfilter of the same frequency band as that of a specific frequencycomponent extraction unit incorporated in the aforementioned digitalwatermark embedding apparatus. More specifically, the unit 11 comprisesan HPF (High Pass Filter) having a specific cutoff frequency, or a BPF(Band Pass Filter) having a passband center frequency. The specificfrequency component extraction unit 11 extracts a specific frequencycomponent, such as a relatively high frequency component, from the inputimage signal 10.

The specific frequency component signal is subjected to phase control ofa predetermined phase control amount by a phase controller 12, i.e., thesignal is phase-shifted. The phase controller 12 is, for example, adigital phase shifter. In the phase shift example in FIG. 2 using thephase controller 12, the phase of the specific frequency componentsignal is shifted with its original waveform maintained. The amount ofphase shift is controlled continuously or stepwise.

The phase-controlled specific frequency component signal is supplied tothe first input of the correlator 13. The correlator 13 computes across-correlation value between the phase-controlled specific frequencycomponent signal and the input image signal 10. The cross-correlationvalue is output from the correlator 13 to a watermark informationestimation unit 14.

The watermark information estimation unit 14 searches for a peak in thecross-correlation value (signal), as shown in FIG. 3, thereby detectingwatermark information by estimation. In the cross-correlation value withrespect to the phase shift amount of the phase controller 12, a peakappears at a certain phase-shift amount. The polarity of the peakindicates the presence of watermark information. If the input imagesignal 10 is under an attack of scaling, the phase shift amount of aspecific frequency component contained in the input image signal 10differs from the amount of phase shift performed on the specificfrequency component by the digital watermark embedding apparatus.

In light of this, in the embodiment, the phase shift amount in the phasecontroller 12 is varied continuously or stepwise, thereby enabling thewatermark information estimation unit 14 to search for peak in thecross-correlation value (signal) output from the correlator 13. Presenceof watermark information is determined from the polarity of the peakthat has been discovered. The peak in the cross-correlation value(signal) assumes a positive or negative polarity in accordance with thevalue between watermark information. In the example of FIG. 3, if thepolarity of the peak is positive, it is determined that the watermarkinformation assumes a value of “1”, while if the polarity of the peak isnegative, it is determined that the watermark information assumes avalue of “0”. Thus, the watermark information estimation unit 14 outputsdetected watermark information 15.

As described above, in the first embodiment, a specific frequencycomponent signal is extracted from an input image signal and subjectedto phase control. The cross-correlation value between thephase-controlled specific frequency component signal and the input imagesignal is computed, whereby watermark information is detected from thecross-correlation value. Watermark information can be easily detectedfrom an input image signal against which an attack of scaling was made,by searching for a peak in the cross-correlation value, while varyingthe amount of phase control.

Second Embodiment

Referring to FIG. 4, a digital watermark detection apparatus accordingto a second embodiment receives, via a recording medium or transmissionmedium, an image signal (input image signal 10) generated by a digitalwatermark embedding apparatus (not shown), as in the first embodiment.The input image signal 10 is supplied to the input of the phasecontroller 12 and the first input of the correlator 13. The image signalsubjected to phase control by the phase controller 12 is supplied to thesecond input of the correction unit 13, where a correlation operation isperformed on the image signal and input image signal 10, therebycomputing an auto-correlation function. The auto-correlation function isinput to a specific frequency component extraction unit 16.

The specific frequency component extraction unit 16 comprises an HPF ora BPF, as in the specific frequency component extraction unit 11 shownin FIG. 1, and extracts a specific frequency component by filtering theauto-correlation function. The extracted specific frequency componentsignal is input to a watermark information estimation unit 17, where thepeak level of the extracted specific frequency component signal issearched for, and the polarity of the peak level is determined.

The peak level of the extracted specific frequency component signalassumes a positive or negative polarity in accordance with the value ofwatermark information embedded in the input image signal 10. If thepolarity of the peak level is positive, a watermark informationestimation unit 17 estimates that the watermark information assumes avalue of “1”, while if the polarity of the peak level is negative, theunit 17 estimates that the watermark information assumes a value of “0”.Thus, the watermark information estimation unit 17 outputs detectedwatermark information 15. If phase control by the phase controller 12, acorrelation operation by the correlator 13 and filtering by the specificfrequency component extraction unit 17 are linear operations, thedigital watermark detection apparatus of the second embodiment isequivalent to that shown in FIG. 1.

In the second embodiment, the auto-correlation function of an inputimage signal is computed and filtered to generate a specific frequencycomponent signal. Watermark information can be easily detected from aninput image signal against which an attack of scaling was made, bycomputing the auto-correlation function of the input image signal whilevarying the amount of phase control with respect to the input imagesignal, searching for the peak level of the specific frequency componentsignal, and determining the polarity of the peak level.

Third Embodiment

Referring to FIG. 5, a digital watermark detection apparatus accordingto a third embodiment will be described. The digital watermark detectionapparatus of FIG. 5 receives, via a recording medium or transmissionmedium, an image signal (input image signal 10) generated by a digitalwatermark embedding apparatus (not shown) as in the first embodiment.The input image signal 10 is supplied to the input of the phasecontroller 12 and the first input of the correlator 13. The image signalsubjected to phase control by the phase controller 12 is supplied to thesecond input of the correction unit 13, where a correlation operation isperformed on the image signal and input image signal 10, therebycomputing an auto-correlation function. The process so far is similar tothat employed in the second embodiment.

In the third embodiment, the auto-correlation function from thecorrelator 13 is input to a first accumulator 20. The first accumulator20 accumulates the auto-correlation function for a first short period oftime corresponding to several lines, one field, several fields, oneframe, or several frames, in which the characteristics of an imagecorresponding to the input image signal does not significantly change,thereby generating a first accumulation signal. The accumulator 20 isreset each time the first accumulation signal is generated, and resumesaccumulation of the auto-correlation function.

The first accumulation signal is input to a specific frequency componentextraction unit 21, where it is filtered. As a result, a specificfrequency component signal is extracted. The specific frequencycomponent signal is input to a normalization unit 22. The normalizationunit 22 normalizes the amplitude of the specific frequency componentsignal so that the characteristics of the image corresponding to theinput image signal 10 do not influence the detection of watermarkinformation. The normalized specific frequency component signal is inputto a second accumulator 23.

The second accumulator 23 accumulates the normalized specific frequencycomponent signal for a second period of time, thereby generating asecond accumulation signal. The second period of time is set to, forexample, 15 sec., 30 sec., or 1 min., which is longer than the firstperiod of time as the accumulation period of the first accumulator 20.The accumulator 23 is reset each time the second accumulation signal isgenerated, and resumes accumulation of the normalized specific frequencycomponent signal. The second accumulation signal is input to a watermarkinformation estimation unit 24, where the peak level of the specificfrequency component signal is searched for, and the polarity of the peaklevel of the second accumulation signal is determined, thereby detectingwatermark information 15.

In the third embodiment, the auto-correlation function of an input imagesignal is computed and accumulated, thereby extracting a specificfrequency component signal. The specific frequency component signal isnormalized in amplitude and accumulated, and watermark information isdetected from the accumulated, normalized specific frequency componentsignal. The auto-correlation function of an input image signal iscomputed while varying the amount of phase control with respect to theinput image signal, thereby searching for the peak level of a specificfrequency component signal and determining the polarity of the peaklevel. By virtue of this process, watermark information can be easilydetected from an input image signal against which an attack of scalingwas made. In this embodiment, since the auto-correlation function isaccumulated and filtered by the specific frequency component extractionunit 21, the number of filtering operations can be reduced, compared tothe case where the cross-correlation value between an input image signaland a filtered image signal is accumulated. Accordingly, the costrequired for detecting watermark information can be reduced withoutdegrading the watermark information detection performance.

Fourth Embodiment

FIG. 6 shows an essential part of a digital watermark detectionapparatus according to a fourth embodiment of the invention. Thisdigital watermark detection apparatus incorporates an operation amountcontroller 25, in addition to the elements employed in the digitalwatermark detection apparatus of FIG. 5. In this embodiment, it isassumed that part or all of the processes of the digital watermarkdetection apparatus is realized by software processing, using aprocessor, such as a versatile or dedicated CPU (Central ProcessingUnit), DSP (Digital Signal Processor), etc. The operation amountcontroller 25 acquires, from, for example, an OS (Operating System),information 26 indicative of the throughput of the processor.“Throughput” means the original performance of the processor and/or theever-changing performance of the processor.

If the throughput of the processor is relatively low, the operationamount controller 25 controls the correlator 13 so as to reduce theoperation amount of the unit 13 per unit time. Specifically, if thethroughput is lower than a predetermined threshold value, the operationamount controller 25 periodically stops the operation of the correlator13 in units of pixels, lines, fields or frames of the input image signal10.

If the operation amount of the correlator 13 is reduced, theaccumulation amount of the specific frequency component signal at thesecond accumulator 23 reduces. Accordingly, the watermark informationdetection performance degrades. To secure the accumulation amount, theoperation amount controller 25 controls the accumulation period (secondperiod) of the accumulator 23. If, for example, the correlator 13 isstopped every two lines to perform a correlation operation every twolines, the operation amount per unit time is halved, accordingly theaccumulation amount of the correlation value is halved. To secure thesame accumulation amount as that obtained when the operation amount ofthe correlator 13 is not controlled, the operation amount controller 25doubles the accumulation period of the accumulator 23.

This enables watermark information to be detected without applying anexcessive load on the processor. Therefore, watermark informationdetection can be realized even if a low-performance processor is used,or the processor is also used for a process other than digital watermarkdetection, which would drop the processor throughput below the thresholdvalue. Conversely, if the throughput of the processor is higher thanrequired, the frequency of stopping the correlation operation can bereduced to increase the accumulation amount and enhance the performanceof watermark information detection.

Fifth Embodiment

FIG. 7 shows an essential part of a digital watermark detectionapparatus according to a fifth embodiment of the invention. In the fifthembodiment, to detect a digital watermark embedded in an image signalsubjected to rotational transform, an image rotation unit 27 forrotating an image corresponding to the input image signal 10 is providedbefore the phase controller and correlator, which differs from the thirdembodiment. The image rotation unit 27 outputs an image signalcorresponding to an image obtained by rotating, in accordance withrotation angle information 28, the image corresponding to the inputimage signal 10. As a result, even if the input image signal 10 is underan attack of rotation, watermark information can be acquired therefrom.

As shown in, for example, FIG. 8, the image rotation unit 27 comprises aline buffer 29 and read unit 30. The line buffer 29 reads andtemporarily stores a plurality of line components of the input imagesignal 10. The line components contained in the image signal stored inthe line buffer 29 are read by the read unit 30, with the reading startportions of the line components being shifted to one another inaccordance with the rotation angle information 28. The read unit 30 setsa line shift amount corresponding to the rotation angle information 29.

FIG. 9 shows the arrangement of image pixels 114 corresponding to theinput image signal 10. In the correlator incorporated in a usual digitalwatermark detection apparatus, a correlation operation is performed in aline direction indicated by reference numeral 111. On the other hand, inthis embodiment, concerning an input image signal under an attack ofrotation, the image rotation unit 27 shifts each line component input tothe correlator 13 in units of predetermined numbers of pixels, asindicated by reference numeral 112. As a result, image signalcomponents, which correspond to pixels 113 expressed by black dots andarranged in an oblique direction as shown in FIG. 9, are sequentiallyinput to the correlator 13, whereby a correlation operation is performedon the pixels in the oblique direction.

If the amount of line shifting in the read unit 30 according to therotation angle information 28 is changed at a position corresponding tointegral multiples of a predetermined number n (e.g., eight) of pixels,for example, at the position of the pixel 113 in FIG. 8, image signaldata in the line buffer 29 can be effectively accessed. Accordingly,even if the input image signal 10 is under an attack of high-speedrotation, digital watermark information can be detected.

Since the rotational angle θ of an image is as small as 0 (θ≈0), cosθ≈1, sin θ≈ tan θ≈θ, line shifting of an input image signal 10 under anattack of rotation input to the correlator 13, performed by the imagerotation unit 27 shown in FIG. 8, enables watermark information to bedetected from the input image signal without increasing the operationamount.

As described above, in this embodiment, the line components of the inputimage signal 10 input to the correlator 13 are gradually shifted,thereby approximating the rotation of the image. In particular, if theimage rotation unit 27 comprises the line buffer 29 and read unit 30 asshown in FIG. 8, watermark information embedded in an input image signalunder an attack of rotation can be detected simply by changing a readaddress in the read unit 30 to change the amount of line shifting.Therefore, increases in the operation amount, the memory bandwidth ofthe line buffer 29 and the entire circuit scale can be avoided.Furthermore, when the position at which the amount of line shifting ischanged is made to correspond to the word width in the line buffer 29,the efficiency of memory access can be enhanced, therefore watermarkinformation can be easily detected even if the input image signal 10 isunder an attack of high-speed rotation.

EXAMPLE 1 OF WATERMARK INFORMATION ESTIMATION UNIT

Referring to FIG. 10, a description will be given of an example of thewatermark information estimation unit 24 in the digital watermarkdetection apparatus of FIG. 5. In this example, the watermarkinformation estimation unit 24 comprises a threshold-setting unit 31,watermark detector 32 and watermark determination unit 33.

The threshold-setting unit 31 acquires information indicative of asecond period of time corresponding to the accumulation period of timeof the second accumulator 23 shown in FIG. 5, thereby changing, based onthe accumulation period, the threshold value set in the watermarkdetermination unit 33 for determining watermark information.Specifically, the longer the accumulation period, the lower thethreshold value is set. The watermark detector 32 detects watermarkinformation from a second accumulation signal (obtained by normalizingand accumulating the specific frequency component of an auto-correlationsignal) output from the second accumulator 23, thereby providing thewatermark determination unit 33 with the detected watermark informationand level (the absolute value of the amplitude of the peak of the secondaccumulation signal).

The watermark determination unit 33 compares the level supplied from thewatermark detector 32 with the threshold value set by thethreshold-setting unit 31. If the level is not less than the thresholdvalue, the watermark determination unit 33 determines that the watermarkdetector 32 has correctly detected watermark information, and outputsthe detected watermark information. If, on the other hand, the level isless than the threshold value, the watermark determination unit 33determines that no watermark information is embedded, and outputs amessage “No Watermark”. As mentioned above, basically, the longer theaccumulation period, the lower the threshold value. However, thethreshold value may also be set higher. The watermark determination unit33 may perform determination in units of predetermined periods (e.g., 15sec., 30 sec., one minute, etc.) using a threshold value correspondingto the period, or may perform determination using a continuously variedthreshold value.

As described above, in the embodiment, when the accumulation period isset long, the threshold value for determining watermark information islowered to increase the probability of detection of watermarkinformation. Accordingly, the detection performance is enhanced withoutincreasing the operation amount or circuit scale required for thedetection of watermark information.

EXAMPLE 2 OF WATERMARK INFORMATION ESTIMATION UNIT

Referring to FIG. 12, a description will be given of another example ofthe watermark information estimation unit 24 in the digital watermarkdetection apparatus of FIG. 5. In this example, the watermarkinformation estimation unit 24 comprises at least two watermarkdetectors 41A and 41B that employ different watermark detection manners,and watermark determination unit 42. The watermark detectors 41A and 41Bindividually detect watermark information. The watermark determinationunit 42 determines whether the detection results of the detectors 41Aand 41B are identical to each other.

The watermark detector 41A receives a second accumulation signal outputfrom the accumulator 23 and indicative of the normalized and accumulatedspecific frequency component of an auto-correlation signal, then detectswatermark information from the second accumulation signal using a firstdetection manner, and supplies the detection result to the watermarkdetermination unit 42. Similarly, the watermark detector 41B detectswatermark information from the second accumulation signal using a seconddetection manner, and supplies the detection result to the watermarkdetermination unit 42. The watermark determination unit 42 compares thewatermark information items from the watermark detectors 41A and 41B. Ifthey are identical, the watermark determination unit 42 determines thatdigital watermark has been detected, and outputs the detected watermarkinformation. If they are not identical, the unit 42 determines that nodigital watermark is embedded, and outputs a message “No Watermark”.

If, for example, the watermark detector 41A has detected watermarkinformation “A” using the first detection manner, and the watermarkdetector 41B has detected watermark information “A” using the seconddetection manner, the two detection results are identical and hencewatermark information “A” is finally acquired as a detection result. Onthe other hand, if watermark information items “B” and “C” are acquiredby the first and second detection manners, respectively, the twodetection results differ from each other and hence watermark informationcannot be confirmed, with the result that it is determined that nowatermark information is embedded. The same idea as that of thisembodiment can be utilized when three or more detection manners areemployed.

As stated above, the embodiment employs comparison of watermarkinformation items obtained using a plurality of detection manners, whichenables accurate detection of watermark information and reduction of theprobability of erroneous detection.

EXAMPLE 3 OF WATERMARK INFORMATION ESTIMATION UNIT

Referring to FIG. 13, a description will be given of a further exampleof the watermark information estimation unit 24. In this example, inaddition to the second accumulator 23 shown in FIG. 5, a thirdaccumulator 50 is provided before the watermark information estimationunit 24 for accumulating the normalized specific frequency component ofan auto-correlation signal. Further, the watermark informationestimation unit 24 comprises a watermark detector 51, watermarkprovisional detector 52, provisional detection determination unit 53 andwatermark determination unit 54.

The second accumulator 23 accumulates a normalized specific frequencycomponent signal for the second period of time, and supplies a secondaccumulation signal to the watermark detector 51. The watermark detector51 detects watermark information, and supplies a detection result to thewatermark determination unit 54. The third accumulator 50 accumulatesthe normalized specific frequency component signal for a third period oftime that is 1/n (n: an integer higher than 1) of the second period, andoutputs an accumulation signal to the watermark provisional detector 52.

The watermark provisional detector 52 performs provisional detection ofwatermark information, and outputs a provisional detection result to theprovisional detection determination unit 53. After the provisionaldetection determination unit 53 accumulates a number n of provisionaldetection results, and compares them, it supplies the watermarkdetermination unit 54 with a determination result indicative of whetheror not more than half of the number n of provisional detection resultsare identical to each other.

If the watermark determination unit 54 receives, from the provisionaldetermination unit 53, a determination result indicating that more thanhalf of the number n of provisional detection results are identical, itdetermines that watermark information has been detected, and outputs thewatermark information supplied from the watermark detector 51. On theother hand, if the watermark determination unit 54 receives, from theprovisional determination unit 53, a determination result indicatingthat not more than half of the number n of provisional detection resultsare identical, it determines that no watermark information is embedded,and outputs a message “No Watermark”.

Specifically, if the detection period of the watermark provisionaldetector 52 is 10 sec. and n=2, it is determined that watermarkinformation “A” has been provisionally detected within the first fiveseconds, and has also provisionally been detected within the last fiveseconds. In this case, since more than half of the provisional detectionresults are identical to each other, the detection results aredetermined to be valid. As a result, the provisional detectiondetermination unit 53 determines that watermark information is embedded,and the watermark detector 51 outputs the detected watermarkinformation. On the other hand, if it is determined that watermarkinformation “B” has been provisionally detected within the first fiveseconds, and watermark information “C” has been provisionally detectedwithin the last five seconds, more than half of the provisionaldetection results are not identical to each other, thereby determiningthat the detection results are invalid. As a result, it is determinedthat no watermark information is embedded.

As described above, in the embodiment, temporal continuity of watermarkinformation is estimated, which enables watermark information to becorrectly detected, i.e., enables the probability of erroneous detectionof watermark information to be reduced.

(Re: Correlator)

A detailed description will now be given of the correlator 13incorporated in the above-described digital watermark detectionapparatuses. In general, a correlation operation means to sum up themultiplication results of corresponding pixel values contained incertain signals X(n) and Y(n). The cross-correlation value (correlationcoefficient) C of the certain signals X(n) and Y(n) is given by thefollowing equation (1):

$\begin{matrix}{C = {\sum\limits_{n = 0}^{l - 1}{{X(n)} \times {Y(n)}}}} & (1)\end{matrix}$where 1 represents a signal length. In the case of auto-correlation,Y(n)=X(n).

FIG. 14 is a view useful in explaining a general correlation operation.In this operation, multiplication and addition are performed a number oftimes corresponding to the number of pixels, therefore a large number ofoperations are required. To reduce the number of operations, thinning ofpixel values is performed. For example, a block, on which multiplicationand addition are performed, and a block, on which these operations arenot performed, are switched in units of numbers of pixels n, therebyreducing the operation amount (to, for example, 1/n of the conventionalone). As a result, the accuracy of a correlation coefficient is reduced,but is still sufficient for the detection of watermark information.Thus, the operation amount can be effectively reduced. Specifically, ifmultiplication and addition are performed concerning every other pixelas shown in FIG. 15, the correlation coefficient C is given by

$\begin{matrix}{C = {\sum\limits_{n = 0}^{l - 1}\left\{ \begin{matrix}{{X(n)} \times {Y(n)}} & {{{if}\mspace{14mu} n} = {even}} \\0 & {else}\end{matrix} \right.}} & (2)\end{matrix}$

The pixel, on which multiplication is performed, and the pixel, on whichmultiplication is not performed, are exchangeable. As a result, thenumber of operations is half that conventionally required.

Alternatively, multiplication and addition may be performed for thefirst eight pixels, and not for the next eight pixels, as is shown inFIG. 16. If this operation is repeated, the correlation coefficient C isgiven by

$\begin{matrix}{C = {\sum\limits_{n = 0}^{l - 1}\left\{ \begin{matrix}{{X(n)} \times {Y(n)}} & {{{if}\mspace{14mu}{n/8}} = {even}} \\0 & {else}\end{matrix} \right.}} & (3)\end{matrix}$

Also in this case, the pixel, on which multiplication is performed, andthe pixel, on which multiplication is not performed, are exchangeable.As a result, the operation amount is half that conventionally required.

Further, the operation may be modified as shown in FIG. 17 and as givenby formulas (2) and (3), where multiplication and addition are performedconcerning every other pixel of the first eight pixels, but notperformed concerning the next eight pixels. In this case, thecorrelation coefficient C is given by the following formula (4), and theoperation amount is ¼ of the conventional one.

$\begin{matrix}{C = {\sum\limits_{n = 0}^{l - 1}\left\{ \begin{matrix}{{X(n)} \times {Y(n)}} & {{{if}\mspace{14mu}{n/8}} = {{{{even}\&}\mspace{11mu} n} = {even}}} \\0 & {else}\end{matrix} \right.}} & (4)\end{matrix}$

As described above, by performing the correlation operation with pixelvalues thinned, the operation amount and circuit scale required for itcan be effectively reduced without degrading the detection performanceof watermark information.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

1. A computer readable storage medium storing instructions of a computerprogram which when executed by a computer results in performance ofsteps comprising: computing an auto-correlation function of the inputimage signal by a correlator; accumulating the auto-correlation functionfor a first period of time to generate a first accumulation signal by afirst accumulator; extracting a specific frequency component signal fromthe first accumulation signal by an extraction unit; normalizing anamplitude of the specific frequency component signal by a normalizingunit; accumulating the normalized specific frequency component signalfor a second period of time longer than the first period of time togenerate a second accumulation signal by a second accumulator; anddetecting the watermark information from the second accumulation signal,wherein at least one of the correlator, the first accumulator, thenormalizing unit, and the second accumulator includes a processor, andfurther comprising controlling computation of the correlator to reducean operation amount of the correlator per unit time by periodicallystopping computation of the corrlelator if a throughput of the processoris lower than a threshold value.
 2. The computer readable storage mediumaccording to claim 1, wherein the detecting detects the watermarkinformation by determining a polarity of a peak of the specificfrequency component signal.
 3. The computer readable storage mediumaccording to claim 1, wherein the detecting detects the watermarkinformation by determining a level of the second accumulation signalusing a threehold value that is changed in accordance with the secondperiod of time.
 4. The computer readable storage medium according toclaim 1, wherein the detecting detects the watermark information usingat least first and second detection manners, the detecting determiningthat the watermark information is embedded, if the detection resultscoincide with each other.